Packages Used:
library(tidyverse)
library(tidymodels)
library(timetk)
library(skimr)
library(lubridate)
library(janitor)
library(modeltime)
library(cowplot)
library(plotly)
Pulling and cleaning data from Excel sheets:
windcapacity_data <- read_csv("data/panhandlewindcapacity1.csv") %>%
clean_names() %>%
rename(amarillo_or_lubbock = closer_to_a_or_l)
wind_data <- read_csv("data/windsampledata.csv") %>%
clean_names() %>%
select(-market_day, -year) %>%
mutate(
datetime = mdy_hm(datetime),
month = factor(month, levels = c("JANUARY", "FEBRUARY", "MARCH", "APRIL", "MAY", "JUNE", "JULY", "AUGUST", "SEPTEMBER", "OCTOBER", "NOVEMBER", "DECEMBER"))
) %>%
distinct(datetime, .keep_all = TRUE)
#wind_data <- wind_data_prelim[!(wind_data_prelim$year == 2017),] %>%
# select(-year)
Possibility: taking out 2017 data since the exact dates of construction of plants weren’t available. ran it this way (taking out all 2017) and the produced models were less accurate. More data seems to outweigh the cons of slightly inaccurately adjusted data in regards to total possible production, so I ended up keeping all data from 2017 for model training.
-- Data Summary ------------------------
Values
Name wind_data
Number of rows 38444
Number of columns 13
_______________________
Column type frequency:
character 1
factor 1
numeric 10
POSIXct 1
________________________
Group variables None
-- Variable type: character --------------------------------------------------------------------------------------------------------------------------
# A tibble: 1 x 8
skim_variable n_missing complete_rate min max empty n_unique whitespace
* <chr> <int> <dbl> <int> <int> <int> <int> <int>
1 peak_type 0 1 6 7 0 3 0
-- Variable type: factor -----------------------------------------------------------------------------------------------------------------------------
# A tibble: 1 x 6
skim_variable n_missing complete_rate ordered n_unique top_counts
* <chr> <int> <dbl> <lgl> <int> <chr>
1 month 0 1 FALSE 12 JUL: 3720, AUG: 3720, JUN: 3600, SEP: 3600
-- Variable type: numeric ----------------------------------------------------------------------------------------------------------------------------
# A tibble: 10 x 11
skim_variable n_missing complete_rate mean sd p0 p25 p50 p75 p100 hist
* <chr> <int> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <chr>
1 temp_amarillo 155 0.996 59.4 20.0 -9.9 44.1 62 74 109 ▁▃▆▇▂
2 temp_lubbock 60 0.998 62.6 19.4 0 48 64.9 77 111 ▁▃▆▇▂
3 dewpoint_amarillo 182 0.995 40.2 17.4 -16.1 26 41 55.9 73 ▁▃▇▇▇
4 dewpoint_lubbock 60 0.998 42.0 17.5 -16.1 27 44.1 57.9 71.6 ▁▂▇▆▇
5 windspeed_amarillo 197 0.995 14.3 51.2 0 9 12.7 17.3 2237. ▇▁▁▁▁
6 windspeed_lubbock 64 0.998 11.7 5.99 0 8 10.4 15 50.6 ▇▇▂▁▁
7 hour_ending 0 1 12.5 6.92 1 7 13 19 24 ▇▇▆▇▇
8 precipitation_lwc_lubbock 36042 0.0625 0.0686 0.249 0.01 0.01 0.01 0.02 5.54 ▇▁▁▁▁
9 precipitation_lwc_amarillo 36496 0.0507 0.0966 0.307 0.01 0.01 0.02 0.05 4.91 ▇▁▁▁▁
10 gr_panhandle_winddata 3 1.00 1777. 1083. -0.28 784. 1813. 2726. 4104. ▇▆▆▇▂
-- Variable type: POSIXct ----------------------------------------------------------------------------------------------------------------------------
# A tibble: 1 x 7
skim_variable n_missing complete_rate min max median n_unique
* <chr> <int> <dbl> <dttm> <dttm> <dttm> <int>
1 datetime 0 1 2017-06-01 00:00:00 2021-10-19 23:00:00 2019-08-10 23:30:00 38444
There is missing data for most predictors, but not to a great enough extent that it would be a problem if they were simply removed. the exception is precipitation for both Amarillo and Lubbock. Models will be run both with and without the few non-NA values to see if it’s worth keeping/imputing.

The default scale of the histogram of wind speed in Amarillo looks way off. We can see that there is an outlier value in the mid-2000s for wind speed, which has to be an error in measurement or recording.
There were 18 instances where the wind speed readings measured 2236.716, so we arrange wind_data by Amarillo wind speed in descending order then remove the first 18 readings, since we’re removing columns with NA values anyway. With the misinput data removed, the Amarillo wind speed plot looks like this.

Data is split into training and testing sets with an 80/20 split, which is suitable for larger datasets. k-fold cross validation with 5 folds and 3 repeats is used to keep the data from becoming too biased to the training set.
Preprocessing steps:
Near-zero variance filter removes variables that are sparse and unbalanced, meaning variables that may have basically the same value for all observations. I don’t think this was necessary because the data is so varied, but I just kept it because it doesn’t hurt.
Yeo-Johnson transformation reduces skew of variables, which I used on all predictors for temperature, dewpoint temp, and wind speed. It’s helpful for some, but not necessary for other types of models.
Removed datetime variable because this is not a time-series forecasting machine learning model.
All nominal / factor variables are changed to dummy variables (binary) which is better for many models.
Models tested and results (for both optimal hyperparameters and actual model performance):
k-nearest neighbors best model: neighbors = 11, RMSE = 601.17
random forest best model: mtry = 6, min_n = 2, RMSE = 546.40
boosted tree: mtry = 10, min_n = 11, learn_rate = .631, RMSE = 587.55
single-layer neural network: hidden units = 5, penalty = 1.00, RMSE = 836.2845
mars model: number of terms = 81, prod_degree = 2, RMSE = 606.0637
The best performing model by RMSE (and also R^2) was the random forest model. The following is a graph of predicted values from the model and actual values from the dataset (use the slider to zoom into a specific timeframe). Overall, I would say the model does a good job of predicting the trends of the actual data, and part of the model error was in events that could not be predicted. At least a couple times, actual value falls to near-zero or actually zero when the model predicts a higher number, which I would assume is equipment failure or maintainence. The variables used in the model are also ones readily available from public weather forecasting data, which makes it realistic in practical usage for wind generation forecasting. Inclusion of additional relevant variables may further increase model accuracy.
LS0tDQp0aXRsZTogIkdyZWF0ZXIgUGFuaGFuZGxlIFdpbmQgR2VuZXJhdGlvbiBEYXRhIEFuYWx5c2lzIg0Kc3VidGl0bGU6ICJBdXN0aW4gU2hpbm4iDQpvdXRwdXQ6IGh0bWxfbm90ZWJvb2sNCi0tLQ0KUGFja2FnZXMgVXNlZDoNCmBgYHtyIGVjaG89VFJVRX0NCmxpYnJhcnkodGlkeXZlcnNlKQ0KbGlicmFyeSh0aWR5bW9kZWxzKQ0KbGlicmFyeSh0aW1ldGspDQpsaWJyYXJ5KHNraW1yKQ0KbGlicmFyeShsdWJyaWRhdGUpDQpsaWJyYXJ5KGphbml0b3IpDQpsaWJyYXJ5KG1vZGVsdGltZSkNCmxpYnJhcnkoY293cGxvdCkNCmxpYnJhcnkocGxvdGx5KQ0KYGBgDQoNClB1bGxpbmcgYW5kIGNsZWFuaW5nIGRhdGEgZnJvbSBFeGNlbCBzaGVldHM6DQpgYGB7cn0NCndpbmRjYXBhY2l0eV9kYXRhIDwtIHJlYWRfY3N2KCJkYXRhL3BhbmhhbmRsZXdpbmRjYXBhY2l0eTEuY3N2IikgJT4lDQogIGNsZWFuX25hbWVzKCkgJT4lDQogIHJlbmFtZShhbWFyaWxsb19vcl9sdWJib2NrID0gY2xvc2VyX3RvX2Ffb3JfbCkNCg0Kd2luZF9kYXRhIDwtIHJlYWRfY3N2KCJkYXRhL3dpbmRzYW1wbGVkYXRhLmNzdiIpICU+JQ0KICBjbGVhbl9uYW1lcygpICU+JQ0KICBzZWxlY3QoLW1hcmtldF9kYXksIC15ZWFyKSAlPiUNCiAgbXV0YXRlKA0KICAgICAgICAgZGF0ZXRpbWUgPSBtZHlfaG0oZGF0ZXRpbWUpLA0KICAgICAgICAgbW9udGggPSBmYWN0b3IobW9udGgsIGxldmVscyA9IGMoIkpBTlVBUlkiLCAiRkVCUlVBUlkiLCAiTUFSQ0giLCAiQVBSSUwiLCAiTUFZIiwgIkpVTkUiLCAiSlVMWSIsICJBVUdVU1QiLCAiU0VQVEVNQkVSIiwgIk9DVE9CRVIiLCAiTk9WRU1CRVIiLCAiREVDRU1CRVIiKSkNCiAgICAgICAgICkgJT4lDQogIGRpc3RpbmN0KGRhdGV0aW1lLCAua2VlcF9hbGwgPSBUUlVFKQ0KDQojd2luZF9kYXRhIDwtIHdpbmRfZGF0YV9wcmVsaW1bISh3aW5kX2RhdGFfcHJlbGltJHllYXIgPT0gMjAxNyksXSAlPiUNCiMgIHNlbGVjdCgteWVhcikNCg0KYGBgDQoNClBvc3NpYmlsaXR5OiB0YWtpbmcgb3V0IDIwMTcgZGF0YSBzaW5jZSB0aGUgZXhhY3QgZGF0ZXMgb2YgY29uc3RydWN0aW9uIG9mIHBsYW50cyB3ZXJlbid0IGF2YWlsYWJsZS4gcmFuIGl0IHRoaXMgd2F5ICh0YWtpbmcgb3V0IGFsbCAyMDE3KSBhbmQgdGhlIHByb2R1Y2VkIG1vZGVscyB3ZXJlIGxlc3MgYWNjdXJhdGUuIE1vcmUgZGF0YSBzZWVtcyB0byBvdXR3ZWlnaCB0aGUgY29ucyBvZiBzbGlnaHRseSBpbmFjY3VyYXRlbHkgYWRqdXN0ZWQgZGF0YSBpbiByZWdhcmRzIHRvIHRvdGFsIHBvc3NpYmxlIHByb2R1Y3Rpb24sIHNvIEkgZW5kZWQgdXAga2VlcGluZyBhbGwgZGF0YSBmcm9tIDIwMTcgZm9yIG1vZGVsIHRyYWluaW5nLg0KDQpgYGB7ciBlY2hvPUZBTFNFfQ0KIyMjIEVEQQ0Kc2tpbSh3aW5kX2RhdGEpDQpgYGANCg0KVGhlcmUgaXMgbWlzc2luZyBkYXRhIGZvciBtb3N0IHByZWRpY3RvcnMsIGJ1dCBub3QgdG8gYSBncmVhdCBlbm91Z2ggZXh0ZW50IHRoYXQgaXQgd291bGQgYmUgYSBwcm9ibGVtIGlmIHRoZXkgd2VyZSBzaW1wbHkgcmVtb3ZlZC4gdGhlIGV4Y2VwdGlvbiBpcyBwcmVjaXBpdGF0aW9uIGZvciBib3RoIEFtYXJpbGxvIGFuZCBMdWJib2NrLiBNb2RlbHMgd2lsbCBiZSBydW4gYm90aCB3aXRoIGFuZCB3aXRob3V0IHRoZSBmZXcgbm9uLU5BIHZhbHVlcyB0byBzZWUgaWYgaXQncyB3b3J0aCBrZWVwaW5nL2ltcHV0aW5nLg0KDQoNCmBgYHtyfQ0Kd2luZF9kYXRhIDwtIG5hLm9taXQod2luZF9kYXRhKQ0Kd2luZF9kYXRhIDwtIHdpbmRfZGF0YSAlPiUNCiAgICAgICAgICAgICBzZWxlY3QoLWNvbnRhaW5zKCJwcmVjaXAiKSkNCg0KcDEgPC0gd2luZF9kYXRhICU+JQ0KICBwbG90X3RpbWVfc2VyaWVzKGRhdGV0aW1lLCBncl9wYW5oYW5kbGVfd2luZGRhdGEsIC50aXRsZSA9ICJHcmVhdGVyIFBhbmhhbmRsZSBXaW5kIERhdGEgb3ZlciBUaW1lIiwgLnNtb290aCA9IEZBTFNFLCAucGxvdGx5X3NsaWRlciA9IFRSVUUpDQoNCnAyIDwtIHdpbmRfZGF0YSAlPiUNCiAgZ2dwbG90KGFlcyhncl9wYW5oYW5kbGVfd2luZGRhdGEpKSArDQogIGdlb21faGlzdG9ncmFtKGJpbnMgPSAzMCkNCg0KIyBhY2NlcHRhYmxlDQpwMyA8LSB3aW5kX2RhdGEgJT4lDQogIGdncGxvdChhZXModGVtcF9hbWFyaWxsbykpICsNCiAgZ2VvbV9oaXN0b2dyYW0oYmlucyA9IDMwKQ0KDQojIGFjY2VwdGFibGUNCnA0IDwtIHdpbmRfZGF0YSAlPiUNCiAgZ2dwbG90KGFlcyh0ZW1wX2x1YmJvY2spKSArDQogIGdlb21faGlzdG9ncmFtKGJpbnMgPSAzMCkNCg0KIyBuZWVkcyB0cmFuc2Zvcm1hdGlvbj8NCnA1IDwtIHdpbmRfZGF0YSAlPiUNCiAgZ2dwbG90KGFlcyhkZXdwb2ludF9hbWFyaWxsbykpICsNCiAgZ2VvbV9oaXN0b2dyYW0oYmlucyA9IDMwKQ0KDQojIG5lZWRzIHRyYW5zZm9ybWF0aW9uPw0KcDYgPC0gd2luZF9kYXRhICU+JQ0KICBnZ3Bsb3QoYWVzKGRld3BvaW50X2x1YmJvY2spKSArDQogIGdlb21faGlzdG9ncmFtKGJpbnMgPSAzMCkNCg0KcDcgPC0gd2luZF9kYXRhICU+JQ0KICBnZ3Bsb3QoYWVzKHdpbmRzcGVlZF9hbWFyaWxsbykpICsNCiAgZ2VvbV9oaXN0b2dyYW0oYmlucyA9IDMwKQ0KDQpwOCA8LSB3aW5kX2RhdGEgJT4lDQogIGdncGxvdChhZXMod2luZHNwZWVkX2x1YmJvY2spKSArDQogIGdlb21faGlzdG9ncmFtKGJpbnMgPSAzMCkNCg0KcGxvdF9ncmlkKHAyLHAzLHA0LHA1LHA2LHA3LHA4KQ0KDQojIyMgYm90aCBkZXdwb2ludHMgYXBwZWFyIHRvIGJlIGJpbW9kYWxseSBkaXN0cmlidXRlZA0KYGBgDQpUaGUgZGVmYXVsdCBzY2FsZSBvZiB0aGUgaGlzdG9ncmFtIG9mIHdpbmQgc3BlZWQgaW4gQW1hcmlsbG8gbG9va3Mgd2F5IG9mZi4gV2UgY2FuIHNlZSB0aGF0IHRoZXJlIGlzIGFuIG91dGxpZXIgdmFsdWUgaW4gdGhlIG1pZC0yMDAwcyBmb3Igd2luZCBzcGVlZCwgd2hpY2ggaGFzIHRvIGJlIGFuIGVycm9yIGluIG1lYXN1cmVtZW50IG9yIHJlY29yZGluZy4NCg0KVGhlcmUgd2VyZSAxOCBpbnN0YW5jZXMgd2hlcmUgdGhlIHdpbmQgc3BlZWQgcmVhZGluZ3MgbWVhc3VyZWQgMjIzNi43MTYsIHNvIHdlIGFycmFuZ2Ugd2luZF9kYXRhIGJ5IEFtYXJpbGxvIHdpbmQgc3BlZWQgaW4gZGVzY2VuZGluZyBvcmRlciB0aGVuIHJlbW92ZSB0aGUgZmlyc3QgMTggcmVhZGluZ3MsIHNpbmNlIHdlJ3JlIHJlbW92aW5nIGNvbHVtbnMgd2l0aCBOQSB2YWx1ZXMgYW55d2F5LiBXaXRoIHRoZSBtaXNpbnB1dCBkYXRhIHJlbW92ZWQsIHRoZSBBbWFyaWxsbyB3aW5kIHNwZWVkIHBsb3QgbG9va3MgbGlrZSB0aGlzLg0KYGBge3IgZWNobz1UUlVFfQ0KIyMjIFRoZSBkZWZhdWx0IHNjYWxlIG9mIHRoZSBoaXN0b2dyYW0gb2Ygd2luZCBzcGVlZCBpbiBBbWFyaWxsbyBsb29rcyB3YXkgb2ZmLiBXZSBjYW4gc2VlIHRoYXQgdGhlcmUgaXMgYW4gb3V0bGllciB2YWx1ZSBpbiB0aGUgbWlkLTIwMDBzIGZvciB3aW5kIHNwZWVkLCB3aGljaCBoYXMgdG8gYmUgYW4gZXJyb3IgaW4gbWVhc3VyZW1lbnQgb3IgcmVjb3JkaW5nLg0KDQp3aW5kX2RhdGEgPC0gd2luZF9kYXRhICU+JSBhcnJhbmdlKGRlc2Mod2luZHNwZWVkX2FtYXJpbGxvKSkNCg0Kd2luZF9kYXRhDQojIyMgdGhlcmUgd2VyZSAxOCBpbnN0YW5jZXMgd2hlcmUgdGhlIHJlYWRpbmdzIG1lYXN1cmVkIDIyMzYuNzE2LCBzbyB3ZSBhcnJhbmdlIHdpbmRfZGF0YSBieSBBbWFyaWxsbyB3aW5kIHNwZWVkIGluIGRlc2NlbmRpbmcgb3JkZXIgdGhlbiByZW1vdmUgdGhlIGZpcnN0IDIwIHJlYWRpbmdzLCBzaW5jZSB3ZSdyZSByZW1vdmluZyBjb2x1bW5zIHdpdGggTkEgdmFsdWVzIGFueXdheS4NCndpbmRfZGF0YSA8LSB3aW5kX2RhdGFbMTg6MzgxODYsXQ0KDQojIyMgdXBkYXRlZCBBbWFyaWxsbyB3aW5kIHNwZWVkIHBsb3QNCndpbmRfZGF0YSAlPiUNCiAgZ2dwbG90KGFlcyh3aW5kc3BlZWRfYW1hcmlsbG8pKSArDQogIGdlb21faGlzdG9ncmFtKGJpbnMgPSAzMCkNCg0KcGxvdF9ncmlkKHAyLHAzLHA0LHA1LHA2LHA5LHA4KQ0KDQpgYGANCkRhdGEgaXMgc3BsaXQgaW50byB0cmFpbmluZyBhbmQgdGVzdGluZyBzZXRzIHdpdGggYW4gODAvMjAgc3BsaXQsIHdoaWNoIGlzIHN1aXRhYmxlIGZvciBsYXJnZXIgZGF0YXNldHMuIGstZm9sZCBjcm9zcyB2YWxpZGF0aW9uIHdpdGggNSBmb2xkcyBhbmQgMyByZXBlYXRzIGlzIHVzZWQgdG8ga2VlcCB0aGUgZGF0YSBmcm9tIGJlY29taW5nIHRvbyBiaWFzZWQgdG8gdGhlIHRyYWluaW5nIHNldC4NCg0KUHJlcHJvY2Vzc2luZyBzdGVwczoNCg0KTmVhci16ZXJvIHZhcmlhbmNlIGZpbHRlciByZW1vdmVzIHZhcmlhYmxlcyB0aGF0IGFyZSBzcGFyc2UgYW5kIHVuYmFsYW5jZWQsIG1lYW5pbmcgdmFyaWFibGVzIHRoYXQgbWF5IGhhdmUgYmFzaWNhbGx5IHRoZSBzYW1lIHZhbHVlIGZvciBhbGwgb2JzZXJ2YXRpb25zLiBJIGRvbid0IHRoaW5rIHRoaXMgd2FzIG5lY2Vzc2FyeSBiZWNhdXNlIHRoZSBkYXRhIGlzIHNvIHZhcmllZCwgYnV0IEkganVzdCBrZXB0IGl0IGJlY2F1c2UgaXQgZG9lc24ndCBodXJ0Lg0KDQpZZW8tSm9obnNvbiB0cmFuc2Zvcm1hdGlvbiByZWR1Y2VzIHNrZXcgb2YgdmFyaWFibGVzLCB3aGljaCBJIHVzZWQgb24gYWxsIHByZWRpY3RvcnMgZm9yIHRlbXBlcmF0dXJlLCBkZXdwb2ludCB0ZW1wLCBhbmQgd2luZCBzcGVlZC4gSXQncyBoZWxwZnVsIGZvciBzb21lLCBidXQgbm90IG5lY2Vzc2FyeSBmb3Igb3RoZXIgdHlwZXMgb2YgbW9kZWxzLg0KDQpSZW1vdmVkIGRhdGV0aW1lIHZhcmlhYmxlIGJlY2F1c2UgdGhpcyBpcyBub3QgYSB0aW1lLXNlcmllcyBmb3JlY2FzdGluZyBtYWNoaW5lIGxlYXJuaW5nIG1vZGVsLg0KDQpBbGwgbm9taW5hbCAvIGZhY3RvciB2YXJpYWJsZXMgYXJlIGNoYW5nZWQgdG8gZHVtbXkgdmFyaWFibGVzIChiaW5hcnkpIHdoaWNoIGlzIGJldHRlciBmb3IgbWFueSBtb2RlbHMuDQoNCmBgYHtyIGVjaG89VFJVRX0NCiMjIHNwbGl0dGluZyB0aGUgZGF0YS4gdGhlIGRhdGFzZXQgaXMgbGFyZ2UsIHNvIHdlIGNhbiB1c2UgYSBwcm9wb3J0aW9uIGxpa2UgODAvMjAgZm9yIHRyYWluaW5nL3Rlc3RpbmcNCnNwbGl0IDwtIGluaXRpYWxfc3BsaXQod2luZF9kYXRhLCBwcm9wID0gLjgsIHN0cmF0YSA9IGdyX3BhbmhhbmRsZV93aW5kZGF0YSkNCg0Kd2luZF9jdiA8LSB2Zm9sZF9jdih0cmFpbmluZyhzcGxpdCksIHYgPSA1LCByZXBlYXRzID0gMywgc3RyYXRhID0gZ3JfcGFuaGFuZGxlX3dpbmRkYXRhKQ0KDQojIyNiYXNpYyByZWNpcGUgc2tlbGV0b24sIHByZWNpcGl0YXRpb24gcmVtb3ZlZA0KcmVjaXBlIDwtIHRyYWluaW5nKHNwbGl0KSAlPiUNCiAgcmVjaXBlKGdyX3BhbmhhbmRsZV93aW5kZGF0YSB+IC4pICU+JQ0KICBzdGVwX256dihhbGxfcHJlZGljdG9ycygpKSAlPiUNCiAgc3RlcF9ZZW9Kb2huc29uKGNvbnRhaW5zKCJ0ZW1wIiksIGNvbnRhaW5zKCJkZXdwb2ludCIpLCBjb250YWlucygid2luZHNwZWVkIikpICU+JQ0KICBzdGVwX3JtKGRhdGV0aW1lLCBjb250YWlucygicHJlY2lwaXRhdGlvbiIpLCBza2lwID0gVFJVRSkgJT4lDQogIHN0ZXBfZHVtbXkoYWxsX25vbWluYWwoKSwgb25lX2hvdCA9IFRSVUUpDQoNCmJha2UocHJlcChyZWNpcGUpLCBuZXdfZGF0YSA9IHRyYWluaW5nKHNwbGl0KSkNCg0KYGBgDQoNCmBgYHtyfQ0KIzEuIGxpbmVhciAgcmVncmVzc2lvbiBtb2RlbA0KbGluZWFyX21vZGVsIDwtIGxpbmVhcl9yZWcobW9kZSA9ICJyZWdyZXNzaW9uIikgJT4lDQogICAgc2V0X2VuZ2luZSgibG0iKQ0KDQojIDIuIG5lYXJlc3QgbmVpZ2hib3JzDQprbm5fbW9kZWwgPC0gbmVhcmVzdF9uZWlnaGJvcihtb2RlID0gInJlZ3Jlc3Npb24iLA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbmVpZ2hib3JzID0gdHVuZSgpKSAlPiUNCiAgc2V0X2VuZ2luZSgia2tubiIpDQoNCiMgMy4gcmFuZG9tIGZvcmVzdA0KcmZfbW9kZWwgPC0gcmFuZF9mb3Jlc3QobW9kZSA9ICJyZWdyZXNzaW9uIiwNCiAgICAgICAgICAgICAgICAgICAgICAgIG1pbl9uID0gdHVuZSgpLA0KICAgICAgICAgICAgICAgICAgICAgICAgbXRyeSA9IHR1bmUoKSkgJT4lIA0KICBzZXRfZW5naW5lKCJyYW5nZXIiKQ0KDQojIDQuIGJvb3N0ZWQgdHJlZQ0KYnRfbW9kZWwgPC0gYm9vc3RfdHJlZShtb2RlID0gInJlZ3Jlc3Npb24iLA0KICAgICAgICAgICAgICAgICAgICAgICBtdHJ5ID0gdHVuZSgpLA0KICAgICAgICAgICAgICAgICAgICAgICBtaW5fbiA9IHR1bmUoKSwNCiAgICAgICAgICAgICAgICAgICAgICAgbGVhcm5fcmF0ZSA9IHR1bmUoKSkgJT4lDQogIHNldF9lbmdpbmUoInhnYm9vc3QiKQ0KDQojIDUuIHN1cHBvcnQgdmVjdG9yIG1hY2hpbmUgKHBvbHlub21pYWwpDQpzdm1wX21vZGVsIDwtIHN2bV9wb2x5KG1vZGUgPSAicmVncmVzc2lvbiIsDQogICAgICAgICAgICAgICAgICAgICAgY29zdCA9IHR1bmUoKSwNCiAgICAgICAgICAgICAgICAgICAgICBkZWdyZWUgPSB0dW5lKCksDQogICAgICAgICAgICAgICAgICAgICAgc2NhbGVfZmFjdG9yID0gdHVuZSgpKSAlPiUNCiAgc2V0X2VuZ2luZSgia2VybmxhYiIpDQoNCiMgNi4gc3VwcG9ydCB2ZWN0b3IgbWFjaGluZSAocmFkaWFsKQ0Kc3Ztcl9tb2RlbCA8LSBzdm1fcmJmKG1vZGUgPSAicmVncmVzc2lvbiIsDQogICAgICAgICAgICAgICAgICAgICAgY29zdCA9IHR1bmUoKSwNCiAgICAgICAgICAgICAgICAgICAgICByYmZfc2lnbWEgPSB0dW5lKCkpICU+JQ0KICBzZXRfZW5naW5lKCJrZXJubGFiIikNCg0KIyA3LiBzaW5nbGUgbGF5ZXIgbmV1cmFsIG5ldHdvcmsNCnNsbm5fbW9kZWwgPC0gbWxwKG1vZGUgPSAicmVncmVzc2lvbiIsDQogICAgICAgICAgICAgICAgICBoaWRkZW5fdW5pdHMgPSB0dW5lKCksDQogICAgICAgICAgICAgICAgICBwZW5hbHR5ID0gdHVuZSgpKSAlPiUNCiAgc2V0X2VuZ2luZSgibm5ldCIpDQoNCiMgOC4gbXVsdGl2YXJpYXRlIGFkYXB0aXZlIHJlZ3Jlc3Npb24gc3BsaW5lcw0KbWFyc19tb2RlbCA8LSBtYXJzKG1vZGUgPSAicmVncmVzc2lvbiIsDQogICAgICAgICAgICAgICAgICAgbnVtX3Rlcm1zID0gdHVuZSgpLA0KICAgICAgICAgICAgICAgICAgIHByb2RfZGVncmVlID0gdHVuZSgpKSAlPiUNCiAgc2V0X2VuZ2luZSgiZWFydGgiKQ0KDQojIyMgY3JlYXRpbmcgcmVndWxhciBncmlkcyBmb3IgdGhlIG1vZGVscy4gNSBsZXZlbHMgZm9yIGVhY2ggaHlwZXItcGFyYW1ldGVyIHNldCBpbiB0aGUgbW9kZWxzIGFib3ZlLg0KIyMjIHJhbmRvbSBmb3Jlc3QgYW5kIGJvb3N0ZWQgdHJlZSBtb2RlbHMgaGF2ZSBtdHJ5IGJldHdlZW4gMiBhbmQgMTAuDQpyZl9wYXJhbWV0ZXJzIDwtIHBhcmFtZXRlcnMocmZfbW9kZWwpICU+JSANCiAgdXBkYXRlKG10cnkgPSBtdHJ5KGMoMiwxMCkpKQ0KDQpidF9wYXJhbXMgPC0gcGFyYW1ldGVycyhidF9tb2RlbCkgJT4lIA0KICB1cGRhdGUobXRyeSA9IG10cnkoYygyLDEwKSksDQogICAgICAgICBsZWFybl9yYXRlID0gbGVhcm5fcmF0ZShjKC01LCAtLjIpKSkNCg0KbWFyc19wYXJhbXMgPC0gcGFyYW1ldGVycyhtYXJzX21vZGVsKSAlPiUgDQogIHVwZGF0ZShudW1fdGVybXMgPSBudW1fdGVybXMoYygyLDMyMCkpKQ0KDQprbm5fZ3JpZCA8LSBncmlkX3JlZ3VsYXIocGFyYW1ldGVycyhrbm5fbW9kZWwpLCBsZXZlbHMgPSA1KQ0KcmZfZ3JpZCA8LSBncmlkX3JlZ3VsYXIocmZfcGFyYW1ldGVycywgbGV2ZWxzID0gNSkNCmJ0X2dyaWQgPC0gZ3JpZF9yZWd1bGFyKGJ0X3BhcmFtcywgbGV2ZWxzID0gNSkNCiMjIyBzdm1wX2dyaWQgPC0gZ3JpZF9yZWd1bGFyKHBhcmFtZXRlcnMoc3ZtcF9tb2RlbCksIGxldmVscyA9IDUpDQojIyMgc3Ztcl9ncmlkIDwtIGdyaWRfcmVndWxhcihwYXJhbWV0ZXJzKHN2bXJfbW9kZWwpLCBsZXZlbHMgPSA1KQ0Kc2xubl9ncmlkIDwtIGdyaWRfcmVndWxhcihwYXJhbWV0ZXJzKHNsbm5fbW9kZWwpLCBsZXZlbHMgPSA1KQ0KbWFyc19ncmlkIDwtIGdyaWRfcmVndWxhcihtYXJzX3BhcmFtcywgbGV2ZWxzID0gNSkNCg0KDQpsaW5lYXJfd29ya2Zsb3cgPC0gd29ya2Zsb3coKSAlPiUgDQogIGFkZF9tb2RlbChsaW5lYXJfbW9kZWwpICU+JSANCiAgYWRkX3JlY2lwZShyZWNpcGUpDQoNCmtubl93b3JrZmxvdyA8LSB3b3JrZmxvdygpICU+JQ0KICBhZGRfbW9kZWwoa25uX21vZGVsKSAlPiUNCiAgYWRkX3JlY2lwZShyZWNpcGUpDQoNCnJmX3dvcmtmbG93IDwtIHdvcmtmbG93KCkgJT4lDQogIGFkZF9tb2RlbChyZl9tb2RlbCkgJT4lDQogIGFkZF9yZWNpcGUocmVjaXBlKQ0KDQpidF93b3JrZmxvdyA8LSB3b3JrZmxvdygpICU+JSANCiAgYWRkX21vZGVsKGJ0X21vZGVsKSAlPiUgDQogIGFkZF9yZWNpcGUocmVjaXBlKQ0KDQpzbG5uX3dvcmtmbG93IDwtIHdvcmtmbG93KCkgJT4lIA0KICBhZGRfbW9kZWwoc2xubl9tb2RlbCkgJT4lIA0KICBhZGRfcmVjaXBlKHJlY2lwZSkNCg0KbWFyc193b3JrZmxvdyA8LSB3b3JrZmxvdygpICU+JQ0KICBhZGRfbW9kZWwobWFyc19tb2RlbCkgJT4lDQogIGFkZF9yZWNpcGUocmVjaXBlKQ0KYGBgDQoNCk1vZGVscyB0ZXN0ZWQgYW5kIHJlc3VsdHMgKGZvciBib3RoIG9wdGltYWwgaHlwZXJwYXJhbWV0ZXJzIGFuZCBhY3R1YWwgbW9kZWwgcGVyZm9ybWFuY2UpOg0KDQprLW5lYXJlc3QgbmVpZ2hib3JzIGJlc3QgbW9kZWw6IG5laWdoYm9ycyA9IDExLCBSTVNFID0gNjAxLjE3DQoNCnJhbmRvbSBmb3Jlc3QgYmVzdCBtb2RlbDogbXRyeSA9IDYsIG1pbl9uID0gMiwgUk1TRSA9IDU0Ni40MA0KDQpib29zdGVkIHRyZWU6IG10cnkgPSAxMCwgbWluX24gPSAxMSwgbGVhcm5fcmF0ZSA9IC42MzEsIFJNU0UgPSA1ODcuNTUNCg0Kc2luZ2xlLWxheWVyIG5ldXJhbCBuZXR3b3JrOiBoaWRkZW4gdW5pdHMgPSA1LCBwZW5hbHR5ID0gMS4wMCwgUk1TRSA9IDgzNi4yODQ1DQoNCm1hcnMgbW9kZWw6IG51bWJlciBvZiB0ZXJtcyA9IDgxLCBwcm9kX2RlZ3JlZSA9IDIsIFJNU0UgPSA2MDYuMDYzNw0KDQpUaGUgYmVzdCBwZXJmb3JtaW5nIG1vZGVsIGJ5IFJNU0UgKGFuZCBhbHNvIFJeMikgd2FzIHRoZSByYW5kb20gZm9yZXN0IG1vZGVsLiBUaGUgZm9sbG93aW5nIGlzIGEgZ3JhcGggb2YgcHJlZGljdGVkIHZhbHVlcyBmcm9tIHRoZSBtb2RlbCBhbmQgYWN0dWFsIHZhbHVlcyBmcm9tIHRoZSBkYXRhc2V0ICh1c2UgdGhlIHNsaWRlciB0byB6b29tIGludG8gYSBzcGVjaWZpYyB0aW1lZnJhbWUpLiBPdmVyYWxsLCBJIHdvdWxkIHNheSB0aGUgbW9kZWwgZG9lcyBhIGdvb2Qgam9iIG9mIHByZWRpY3RpbmcgdGhlIHRyZW5kcyBvZiB0aGUgYWN0dWFsIGRhdGEsIGFuZCBwYXJ0IG9mIHRoZSBtb2RlbCBlcnJvciB3YXMgaW4gZXZlbnRzIHRoYXQgY291bGQgbm90IGJlIHByZWRpY3RlZC4gQXQgbGVhc3QgYSBjb3VwbGUgdGltZXMsIGFjdHVhbCB2YWx1ZSBmYWxscyB0byBuZWFyLXplcm8gb3IgYWN0dWFsbHkgemVybyB3aGVuIHRoZSBtb2RlbCBwcmVkaWN0cyBhIGhpZ2hlciBudW1iZXIsIHdoaWNoIEkgd291bGQgYXNzdW1lIGlzIGVxdWlwbWVudCBmYWlsdXJlIG9yIG1haW50YWluZW5jZS4gVGhlIHZhcmlhYmxlcyB1c2VkIGluIHRoZSBtb2RlbCBhcmUgYWxzbyBvbmVzIHJlYWRpbHkgYXZhaWxhYmxlIGZyb20gcHVibGljIHdlYXRoZXIgZm9yZWNhc3RpbmcgZGF0YSwgd2hpY2ggbWFrZXMgaXQgcmVhbGlzdGljIGluIHByYWN0aWNhbCB1c2FnZSBmb3Igd2luZCBnZW5lcmF0aW9uIGZvcmVjYXN0aW5nLiBJbmNsdXNpb24gb2YgYWRkaXRpb25hbCByZWxldmFudCB2YXJpYWJsZXMgbWF5IGZ1cnRoZXIgaW5jcmVhc2UgbW9kZWwgYWNjdXJhY3kuDQoNCg0KYGBge3IgZWNobz1UUlVFfQ0KIyMjIGFmdGVyIHJ1bm5pbmcgdHVuaW5nIHNjcmlwdA0Kc2hvd19iZXN0KGtubl90dW5lLCBtZXRyaWMgPSAicm1zZSIsIG4gPSA1KQ0Kc2hvd19iZXN0KHJmX3R1bmUsIG1ldHJpYyA9ICJybXNlIiwgbiA9IDUpDQpzaG93X2Jlc3QoYnRfdHVuZSwgbWV0cmljID0gInJtc2UiLCBuID0gNSkNCnNob3dfYmVzdChzbG5uX3R1bmUsIG1ldHJpYyA9ICJybXNlIiwgbiA9IDUpDQpzaG93X2Jlc3QobWFyc190dW5lLCBtZXRyaWMgPSAicm1zZSIsIG4gPSA1KQ0KDQojIyMgcmFuZG9tIGZvcmVzdCBtb2RlbCBpcyBjaG9zZW4gYXMgYmVzdCBvdXQgb2YgdGhlIG90aGVyIG1vZGVscywgYmFzZWQgb24gYm90aCBSXjIgdmFsdWUgYW5kIFJNU0UNCnJmX3dvcmtmbG93X3R1bmVkIDwtIHJmX3dvcmtmbG93ICU+JSANCiAgZmluYWxpemVfd29ya2Zsb3coc2VsZWN0X2Jlc3QocmZfdHVuZSwgbWV0cmljID0gInJtc2UiKSkNCg0KcmZfcmVzdWx0cyA8LSBmaXQocmZfd29ya2Zsb3dfdHVuZWQsIHRyYWluaW5nKHNwbGl0KSkNCg0KeWVzIDwtIHByZWRpY3QocmZfcmVzdWx0cywgbmV3X2RhdGEgPSB0ZXN0aW5nKHNwbGl0KSwgdHlwZSA9ICJudW1lcmljIikgJT4lIA0KICBiaW5kX2NvbHModGVzdGluZyhzcGxpdCkgJT4lIHNlbGVjdChkYXRldGltZSkpDQoNCnJmX3ByZWRzIDwtIHRpYmJsZShEYXRlVGltZSA9IHRlc3Rpbmcoc3BsaXQpJGRhdGV0aW1lLCBQcmVkaWN0ZWQgPSB5ZXMkLnByZWQpDQoNCnJmX3ByZWRzDQoNCmF1dG9wbG90KHJmX3R1bmUsIG1ldHJpYyA9ICJybXNlIikNCg0KY29tYmluYXRpb24gPC0gYmluZF9jb2xzKHRlc3Rpbmcoc3BsaXQpLCByZl9wcmVkcykNCg0KcmZfcHJlZHMgJT4lIHBsb3RfdGltZV9zZXJpZXMoRGF0ZVRpbWUsIFByZWRpY3RlZCkNCg0KZ2dwbG90KCkgKyANCiAgZ2VvbV9saW5lKGRhdGEgPSByZl9wcmVkcywgYWVzKHggPSBEYXRlVGltZSwgeSA9IFByZWRpY3RlZCksIGNvbG9yID0gInJlZCIpICsNCiAgZ2VvbV9saW5lKGRhdGEgPSB0ZXN0aW5nKHNwbGl0KSwgYWVzKHggPSBkYXRldGltZSwgeSA9IGdyX3BhbmhhbmRsZV93aW5kZGF0YSksIGNvbG9yID0gImJsdWUiKQ0KDQpjb21wYXJpc29uIDwtIGNvbWJpbmF0aW9uICU+JQ0KICBzZWxlY3QoZGF0ZXRpbWUsIFByZWRpY3RlZCwgZ3JfcGFuaGFuZGxlX3dpbmRkYXRhKSAlPiUNCiAgcmVuYW1lKCJQcmVkaWN0ZWQgRGF0YSIgPSBQcmVkaWN0ZWQsDQogICAgICAgICAiQWN0dWFsIFZhbHVlcyIgPSBncl9wYW5oYW5kbGVfd2luZGRhdGEpDQoNClRTc3R1ZGlvOjp0c19wbG90KGNvbXBhcmlzb24sDQogICAgICAgICAgICAgICAgICB0aXRsZSA9ICJQcmVkaWN0ZWQgdnMuIEFjdHVhbCBEYXRhIiwNCiAgICAgICAgICAgICAgICAgIHNsaWRlciA9IFRSVUUsDQogICAgICAgICAgICAgICAgICBYZ3JpZCA9IFRSVUUsDQogICAgICAgICAgICAgICAgICBZZ3JpZCA9IFRSVUUNCiAgICAgICAgICAgICAgICAgICkNCmBgYA0KDQoNCg0KDQoNCg0KDQoNCg0KDQoNCg==